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Program Description 1 

Corrective Reading is designed to promote reading accuracy 
(decoding), fluency, and comprehension skills of students 
in grades 4-12 who are reading below their grade level. The 
program includes four sequential levels that address students’ 
decoding skills and six sequential levels that address students’ 
comprehension skills. The levels are designed to target students 
who need assistance with particular types of reading skills 
based on the results of Corrective Reading placement tests (see 

the Teaching section for more details on the different levels). 

The decoding and comprehension components can be used 
separately as a supplemental reading intervention or combined 
for use as a reading intervention curriculum. All lessons in the 
program are sequenced and scripted. Corrective Reading can 
be implemented in small groups of four to five students or in a 
whole-class format. Corrective Reading is intended to be taught 
in 45-minute lessons four to five times a week. 

Research 2 3 

One study of Corrective Reading that falls within the scope of the 
Adolescent Literacy review protocol meets What Works Clear- 
inghouse (WWC) evidence standards, and no studies meet WWC 
evidence standards with reservations. This study included 86 

Based on one study, the WWC considers the extent of evi- 
dence for Corrective Reading on adolescent learners to be small 
for alphabetics, reading fluency, and comprehension. The one 
study that meets WWC evidence standards did not examine the 


fifth-grade struggling readers from a school district just outside 
Pittsburgh, PA. 3 


effectiveness of Corrective Reading on adolescent learners in the 
alphabetic, reading fluency, or general literacy achievement domains. 


1. The descriptive information for this program was obtained from a publicly available source: the program’s website (http://www.sraonline.com/, 
downloaded December 2009). The WWC requests developers to review the program description sections for accuracy from their perspective. Further 
verification of the accuracy of the descriptive information for this program is beyond the scope of this review. The literature search reflects documents 
publicly available by April 2009. 

2. The studies in this report were reviewed using WWC Evidence Standards, Version 2.0 (see the WWC Procedures and Standards Handbook, Chapter III), 
as described in protocol Version 2.0. 

3. The evidence presented in this report is based on available research. Findings and conclusions may change as new research becomes available. 
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Effectiveness 


Corrective Reading was found to have no discernible effects on the alphabetics, reading fluency, and comprehension domains for 
adolescent learners. 


Alphabetics 

Reading fluency 

Comprehension 

General literacy 
achievement 


Rating of effectiveness 

No discernible effects 

No discernible effects 

No discernible effects 

na 

Improvement index 4 

Average: +4 percentile 

+4 percentile points 

Average: +3 percentile 

na 


points 


points 



Range: +1 to +6 

na 

Range: +1 to +5 

na 


percentile points 


percentile points 

na = not applicable 


Absence Of conflict Of The study in this intervention report, Torgesen et al. (2006), was 
interest prepared, in part, by staff of Mathematica Policy Research. For 
this reason, the study was rated by, and this intervention report 


was prepared by, researchers unaffiliated with Mathematica. The 
report was then reviewed by the principal investigator, a WWC 
Quality Assurance reviewer, and an external peer reviewer. 


Additional program 
information 


Developer and contact 

Corrective Reading is distributed by SRA/McGraw-Hill. Address: 
220 East Danieldale Road, Desoto, TX 75115-2490. Web: http:// 
www.sraonline.com/. Telephone: (888) 772-4543. 

Scope of use 

Corrective Reading has been implemented in the United States 
and England. No information is available on the number of 
students or schools using the program. 

Teaching 

The program’s 45-minute lessons are designed for groups of 
up to 20 students up to five times a week. The program’s two 
components— decoding and comprehension— have four and 
six sequential levels of difficulty, respectively. Students’ skill 
development is designed to progress as they move from lower 
to higher levels. For example, a student who needs assistance 
developing basic decoding skills would start at decoding level A 
and complete that level before moving on to the more advanced 


skills covered in level B1, whereas a student who does not read 
fluently or who confuses similar words would start at decoding 
level B1 and complete that level before moving on to the more 
advanced skills covered in level C. 

Each level spans half of an academic year (with the exception 
of level C, which spans an entire academic year, and Fast Cycle 
levels that span roughly a quarter of a year). Therefore, the 
number of levels a student covers in a single academic year can 
range from one to three. The program can be used to provide 
students with either two full periods of instruction per day— one 
period in decoding and one period in comprehension— or one 
period of instruction per day (by focusing on one of the two 
components [e.g., comprehension]). 

Corrective Reading placement tests determine the level at 
which each student is placed; once placed at a particular level, 
the program calls for the student to complete all the lessons in 
that level before moving on to the next level. All levels contain 
ongoing mastery tests and assessments to help track individual 
student achievement. 


4. These numbers show the average and range of student-level improvement indices for all findings across the study. 
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Additional program 
information 

(continued) 


The decoding levels include 

• level A (65 lessons), which is designed for nonreaders. This 
level emphasizes basic decoding skills: rhyming, sounding 
out, sentence reading, and story reading; 

• levels B1 and B2 (65 lessons in each), which are designed 
for struggling readers who do not read fluently or who con- 
fuse similar words. These levels teach students to become 
automatic decoders, able to read 90 words per minute by 
the end of B1 and 130 words per minute by the end of B2; and 

• level C (125 lessons), which is designed for students who 
experience difficulty with vocabulary and complex sentence 
structures. This level bridges the gap between advanced 
word decoding skills and the ability to read informational text. 

The comprehension levels include 

• level A (65 lessons), which is designed for students who do 
not understand the concepts underlying much of the mate- 
rial being taught in classrooms; 

• levels B1 and B2 (60 and 65 lessons, respectively), which 
target more advanced readers; * * * * 5 and 

• level C (140 lessons), which focuses on applying compre- 
hension skills. 


The development of skills in the comprehension component 
progresses from comprehending oral language to comprehend- 
ing written material. Skills are first taught in structured exercises 
that are controlled by the teacher. Later, students are shown how 
to apply the skills independently to complex written materials. 

The publisher also provides staff development training that 
focuses on how to deliver direct instruction and use the program 
materials. Follow-up observations and coaching are recom- 
mended as support for teachers implementing the program. A 
Teaching Tutor CD-ROM provides ongoing support for teachers 
using Corrective Reading. 

Cost 

Prices vary by level (A, B1, B2, C) and component (decoding, 
comprehension). The cost of student materials ranges from $10 
per student for level A programs to $50 per student for level C 
materials. Teacher materials cost approximately $200 per level. 

For more detailed cost information by level and component, 
consult the distributor’s website: http://www.sraonline.com/. 


Research A total of 1 29 studies reviewed by the WWC investigated the 

effects of Corrective Reading on adolescent learners. One study 
(Torgesen et al., 2006) is a randomized controlled trial that meets 
WWC evidence standards. The remaining 128 studies do not 
meet either WWC evidence standards or eligibility screens. 

Meets evidence standards 

Torgesen et al. (2006) conducted a randomized controlled trial 
that examined the effects of the decoding component of Cor- 
rective Reading on 86 fifth-grade students in Pennsylvania. The 


study design was based on random assignment of 32 school 
units 6 to one of four interventions: Corrective Reading, Kaplan 
SpellRead, 7 Failure Free Reading, and Wilson Reading. Within 
each school, eligible students were randomly assigned to the 
treatment group that would receive the intervention assigned to 
its school or to the control group that would not receive any of 
the four interventions. Students were eligible for participation if 
their teacher identified them as a struggling reader and if they 
scored at or below the 30th percentile on a word-level reading 
test and at or above the 5th percentile on a vocabulary test. The 


5. Comprehension levels A and B1 also offer Fast Cycle alternatives that contain 30 and 35 lessons, respectively. 

6. A school unit consists of several schools partnering so that the cluster included two 3rd-grade and two 5th-grade instructional groups. Only the findings 
on 5th graders are included in this review as specified by the Adolescent Literacy review protocol. 

7. The study’s authors refer to the intervention as SpellRead P.A.T. In 2006, Kaplan K12 acquired SpellRead, the developer and distributor of SpellRead 
Phonological Auditory Training®. 
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Research (continued) WWC based its effectiveness ratings on findings from com- 
parisons of the 55 fifth-grade students who received Corrective 
Reading and the 31 fifth-grade control group students who 
received the standard district curriculum. The study reported 
student outcomes after six months of program implementation. 8 

Extent of evidence 

The WWC categorizes the extent of evidence in each domain 
as small or medium to large (see the WWC Procedures and 
Standards Handbook, Appendix G). The extent of evidence 


takes into account the number of studies and the total sample 
size across the studies that meet WWC evidence standards with 
or without reservations. 9 

The WWC considers the extent of evidence for Corrective 
Reading to be small for the alphabetics, reading fluency, and 
comprehension domains for adolescent learners. The one study 
that meets WWC evidence standards did not examine the effect- 
iveness of Corrective Reading on adolescent learners in the alpha- 
betic, reading fluency, or general literacy achievement domains. 


Effectiveness Findings 

The WWC review of interventions for Adolescent Literacy 
addresses student outcomes in four domains: alphabetics, 
reading fluency, comprehension, and general literacy achieve- 
ment. The study included in this report covers three domains: 
alphabetics, reading fluency, and comprehension. The findings 
below present the authors’ estimates and WWC-calculated 
estimates of the size and the statistical significance of the effects 
of Corrective Reading on adolescent learners. 10 

Alphabetics. Torgesen et al. (2006) did not find statistically 
significant effects of Corrective Reading on 5th graders’ scores 
on the Word Attack and Word Identification subtests of the 
Woodcock Reading Mastery Test-Revised (WRMT-R) or the 
Phonemic Decoding Efficiency and Sight Word Efficiency 
subtests of the Test of Word Reading Efficiency (TOWRE). The 
WWC-calculated average effect across these measures was not 


large enough to be considered substantively important accord- 
ing to WWC criteria (i.e., an effect size of at least 0.25). 

Reading fluency. Torgesen et al. (2006) did not find statistically 
significant effects of Corrective Reading on 5th graders’ scores 
on the Oral Reading Fluency test. The WWC-calculated effect 
was not large enough to be considered substantively important 
according to WWC criteria. 

Comprehension. Torgesen et al. (2006) examined two 
outcomes in this domain (the WRMT-R Passage Comprehension 
subtest and the Group Reading Assessment and Diagnostic 
Evaluation [GRADE] Passage Comprehension subtest) and 
reported no statistically significant effects for 5th-grade students, 
although the comprehension component of Corrective Reading 
was not implemented in this study. The WWC-calculated aver- 
age effect across these measures was not large enough to be 
considered substantively important according to WWC criteria. 


8. For the purposes of this study, only the decoding component of Corrective Reading was implemented. By design (to facilitate the examination of two 
types of interventions), the comprehension component of Corrective Reading was not implemented. Additional findings reflecting students’ outcomes 
one year after the intervention year can be found in Appendices A4.1-A4.3. 

9. The extent of evidence categorization was developed to tell readers how much evidence was used to determine the intervention rating, focusing on the 
number and size of studies. Additional factors associated with a related concept— external validity, such as the students’ demographics and the types 
of settings in which studies took place— are not taken into account for the categorization. Information about how the extent of evidence rating was 
determined for Corrective Reading is in Appendix A6. 

10. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within 
classrooms or schools and for multiple comparisons. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures and 
Standards Handbook, Appendix C for clustering and WWC Procedures and Standards Handbook, Appendix D for multiple comparisons. In the case of 
Torgesen et al. (2006), the authors adjusted for clustering and no corrections for multiple comparisons were needed because there were no statistically 
significant findings. 


WWC Intervention Report Corrective Reading 


August 2010 


Effectiveness (continued) In summary, the study showed indeterminate effects in the 

alphabetics, reading fluency, and comprehension domains. 

Rating of effectiveness 

The WWC rates the effects of an intervention in a given outcome 
domain as positive, potentially positive, mixed, no discern- 
ible effects, potentially negative, or negative. The rating of 


effectiveness takes into account four factors: the quality of the 
research design, the statistical significance of the findings, the 
size of the difference between participants in the intervention 
and the comparison conditions, and the consistency in findings 
across studies (see the WWC Procedures and Standards Hand- 
book, Appendix E). 


The WWC found Corrective 
Reading to have no 
discernible effects on 
alphabetics, reading fluency, 
or comprehension for 
adolescent learners 


Improvement index 

The WWC computes an improvement index for each individual 
finding. In addition, within each outcome domain, the WWC 
computes an average improvement index for each study and an 
average improvement index across studies (see WWC Proce- 
dures and Standards Handbook, Appendix F). The improvement 
index represents the difference between the percentile rank 
of the average student in the intervention condition and the 
percentile rank of the average student in the comparison condi- 
tion. Unlike the rating of effectiveness, the improvement index is 
entirely based on the size of the effect, regardless of the statisti- 
cal significance of the effect, the study design, or the analysis. 
The improvement index can take on values between -50 and 
+50, with positive numbers denoting favorable results for the 
intervention group. 


The average improvement index for alphabetics is +4 percen- 
tile points, with a range of +1 to +6 percentile points across find- 
ings from one study. The improvement index for reading fluency 
is +4 percentile points for a single finding from one study. The 
average improvement index for comprehension is +3 percentile 
points, with a range of +1 to +5 percentile points across findings 
from one study. 

Summary 

The WWC reviewed 129 studies on Corrective Reading for 
adolescent learners. One of these studies meets WWC evidence 
standards; the remaining 128 studies do not meet either WWC 
evidence standards or eligibility screens. Based on one study, the 
WWC found no discernible effects on alphabetics, reading flu- 
ency, or comprehension for adolescent learners. The conclusions 
presented in this report may change as new research emerges. 
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Appendix 


Appendix A1 Study characteristics: Torgesen et al. (2006) 


Characteristic 

Description 

Study citation 

Torgesen, J., Myers, D., Schirm, A., Stuart, E., Vartivarian, S., Mansfield, W., et al. (2006). National assessment of Title 1. Interim report. Volume II: Closing the reading gap: 
First year findings from a randomized trial of four reading interventions for striving readers. Washington, DC: National Center for Education Evaluation and Regional Assistance. 
Additional source: Torgesen, J., Schirm, A., Castner, L., Vartivarian, S., Mansfield, W., Myers, D., etal. (2007). National assessment of Title 1. Final report. Volume II: 
Closing the reading gap: Findings from a randomized trial of four reading interventions for striving readers (NCEE 2008-4013). Washington, DC: National Center for Educa- 
tion Evaluation and Regional Assistance. 

Participants 

The study design was based on random assignment of 32 school units, 1 formed from a pool of 52 schools, to one of four interventions ( Corrective Reading, Kaplan SpellRead, 
Failure Free Reading, and Wilson Reading). Within each school, students were randomly assigned to the treatment group that would receive the intervention assigned to its 
school or to the control group that would receive the standard reading curriculum. This report focuses on schools assigned to Corrective Reading and on findings for 5th grad- 
ers (as specified by the Adolescent Literacy review protocol). At the time of the analysis, the sample relevant to this review included 86 fifth-grade students (55 in Corrective 
Reading and 31 in the control group) in seven school units. The number of 5th-grade students at baseline was not reported. 2 Students were eligible for participation if their 
teacher identified them as a struggling reader and if they scored at or below the 30th percentile on a word-level reading test and at or above the 5th percentile on a vocabulary 
test. On average, at baseline, students scored about one-half to one standard deviation below national norms on measures used to assess their ability to decode words. About 
51% of the intervention group students were females, compared to 36% in the control group. About 41% of the intervention group students were eligible for free or reduced- 
price lunch programs, equal to 41% of the students in the control group. 

Setting 

The analysis sample included seven school units in the Allegheny Intermediate Unit (AIU), outside Pittsburgh, Pennsylvania. The AIU consisted of 42 school districts. 

Intervention 

The decoding component of Corrective Reading was implemented by nine teachers beginning in the first week in November 2003 through the first week in May 2004. 

The comprehension component was not implemented. The intervention was administered to students in groups of three that were heterogeneous with regard to students' 
basic reading skills. The average skills of the students in each of the instructional groups determined the pace of instruction. Implementation fidelity was determined by 
reading program trainers who observed the teachers and coached them over a period of months, project coordinators who observed a sample of instructional sessions, 
and ratings based on a sample of videotaped sessions. Implementation was rated as acceptable. The decoding component used in the study included four levels — A, B1, 
B2, and C. Placement testing was used to start each group at the appropriate level. The lessons provided during the study clustered in levels B1 and B2. For those groups 
that progressed to level C, explicit vocabulary instruction was not provided. Over a six-month period, students received a total of about 90 hours of instruction. Students 
received Corrective Reading instruction five days a week in sessions that were approximately 55 minutes long. The study reported student outcomes after six months of 
program implementation. Additional findings reflecting students’ outcomes one year after the end of the implementation of the intervention can be found in Appendices 
A4.1-A4.3. 

Comparison 

The control group students received their regular reading instruction, which included typical classroom instruction and, in many cases, other services (such as another pull-out 
program). Across four interventions, the control group students had fewer small-group instructional hours and average weekly hours of total reading instruction than the 
intervention group students. 


(continued) 
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Appendix A1 Study characteristics: Torgesen et al. (2006) (continued) 


Characteristic 

Description 

Primary outcomes 
and measurement 

The primary outcome measures in the alphabetics domain were the Word Identification and Word Attack subtests of the Woodcock Reading Mastery Test-Revised (WRMT-R) 
and the Phonetic Decoding Efficiency and Sight Word Efficiency subtests of the Test of Word Reading Efficiency (TOWRE). The primary measure in the reading fluency 
domain was the Oral Reading Fluency test (also referred to as AIMSweb). The primary measures in the comprehension domain were the WRMT-R: Comprehension subtest 
and the Group Reading Assessment and Diagnostic Evaluation (GRADE): Passage Comprehension subtest. For a more detailed description of these outcome measures, see 
Appendices A2.1-A2.3. 

Staff/teacher training 

Professional development on how to use Corrective Reading included training and coaching by Corrective Reading program staff, teachers' independent study of program 
materials, and telephone conferences between teachers and Corrective Reading staff. On average, throughout the course of the study, the Corrective Reading intervention 
group teachers participated in 70.8 professional development hours specifically related to using Corrective Reading (32.8 hours were initial training in use of the program, 26.4 
hours were spent in a practice phase, and 11.6 hours occurred during the six-month period in which teachers were using Corrective Reading). 


1. A school unit consists of several schools partnering so that the cluster included two 3rd-grade and two 5th-grade instructional groups. 

2. The study reported that 10 students in the intervention group and no students in the control group were lost to analysis. However, it is not clear if those students were in 5th grade or were part 
of the sample of 3rd-grade students that was also examined in this study. The 3rd-grade sample that was included in this study is not reviewed in this report because it is outside the scope of 
the review. 


WWC Intervention Report Corrective Reading August 2010 




Appendix A2.1 Outcome measures for the alphabetics domain 


Outcome measure 

Description 

Phonics construct 

Woodcock Reading Mastery 
Test-Revised (WRMT-R): 
Word Attack subtest 

This standardized test measures phonemic decoding skills by asking students to pronounce printed pseudo-words. Students are aware that the words are not real (as cited in 
Torgesen et at, 2006). 

Woodcock Reading Mastery 
Test-Revised (WRMT-R): 
Word Identification subtest 

The word identification subtest is a test of decoding skills. The standardized test requires children to pronounce real words from a list of increasing difficulty (as cited in 
Torgesen et at, 2006). 

Test of Word Reading 
Efficiency (TOWRE): 
Phonemic Decoding 
Efficiency (PDE) subtest 

The TOWRE is a standardized, nationally normed measure. The PDE subtest measures the number of nonwords of increasing difficulty that students can pronounce within 45 
seconds (as cited in Torgesen et at, 2006). 

Test of Word Reading 
Efficiency (TOWRE): 
Sight Word Efficiency 
(SWE) subtest 

The TOWRE is a standardized, nationally normed measure. The SWE subtest measures the number of real words of increasing difficulty that students can pronounce within 45 
seconds (as cited in Torgesen et at, 2006). 


Appendix A2.2 Outcome measures for the reading fluency domain 


Outcome measure 

Description 

Oral Reading Fluency 
assessment 

This test (also referred to as AIMSweb) measures the number of words correct per minute (WCPM) that students read using three brief grade-level-appropriate passages. 
These passages contain both fiction and nonfiction text. The norms for this test are updated by Edformation each school year (as cited in Torgesen et at, 2006). 
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Appendix A2.3 Outcome measures for the comprehension domain 


Outcome measure 

Description 

Reading comprehension construct 

Woodcock Reading 
Mastery Test-Revised 
(WRMT-R): Passage 
Comprehension subtest 

In this standardized test, comprehension is measured by having students read silently and fill in missing words in a short paragraph (as cited in Torgesen et al., 2006). 

Group Reading Assessment 
and Diagnostic Evaluation 
(GRADE): Passage 
Comprehension subtest 

The GRADE is a norm-referenced reading assessment that can be used with students at any level. The GRADE has four subtests: (1) Vocabulary, (2) Sentence Comprehen- 
sion, (3) Passage Comprehension, and (4) Listening Comprehension. The Passage Comprehension subtest includes a passage of text and corresponding multiple-choice 
comprehension questions (as cited in Torgesen et al., 2006). 
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Appendix A3.1 Summary of study findings included in the rating for the alphabetics domain 1 





Authors’ findings from the study 







Mean outcome 2 
(standard deviation) 3 


WWC calculations 


Outcome measure 

Study 

sample 

Sample size 
(school units/ 
students) 

Corrective 

Reading Comparison 

group group 

Mean 

difference 4 
(Corrective 
Reading 
- comparison) 

Statistical 

Effect significance 6 

size 5 (at a = 0.05) 

Improvement 

index 7 


Torgesen et al., 2006 8 


WRMT-R: Word 
Attack subtest 

Grade 5 

7/86 

97.40 

(15.00) 

95.50 

(15.00) 

1.90 

0.13 

ns 

+5 

WRMT-R: Word 
Identification subtest 

Grade 5 

7/86 

92.90 

(15.00) 

92.60 

(15.00) 

0.30 

0.02 

ns 

+1 

TOWRE: Phonemic Decoding 
Efficiency (PDE) subtest 

Grade 5 

7/86 

87.30 

(15.00) 

85.40 

(15.00) 

1.90 

0.13 

ns 

+5 

TOWRE: Sight Word 
Efficiency (SWE) subtest 

Grade 5 

7/86 

88.70 

(15.00) 

86.50 

(15.00) 

2.20 

0.15 

ns 

+6 

Domain average for alphabetics (Torgesen et al., 2006) 9 





0.10 

ns 

+4 


ns = not statistically significant 

WRMT-R = Woodcock Reading Mastery Test-Revised 

TOWRE = Test of Word Reading Efficiency 


1. This appendix reports findings considered for the effectiveness rating and the average improvement indices for the alphabetics domain. Follow-up findings from the same studies are not 
included in these ratings but are reported in Appendix A4.1 . Torgesen et al. (2006) also included subgroup analyses by initial skill level (WRMT-R Word Attack subtest and Peabody Picture 
Vocabulary Test [PPVT]) and socioeconomic status. The study reported that Corrective Reading had statistically significant positive effects on TOWRE Sight Word Efficiency for students with 
low initial Word Attack scores and low initial PPVT scores, and for students who were eligible for free or reduced-price lunch programs. The study also found statistically significant positive 
effects on WRMT-R Word Attack scores for students with low initial PPVT scores. No other differences were found between subgroups of students for outcomes in the alphabetics domain. 

2. For Torgesen et al. (2006), the mean outcomes were computed using information reported in the paper. For the control group, the mean outcome is the control group baseline mean standard 
score plus the control group gain. For the intervention group, the mean outcome is the control group baseline mean standard score plus the control group gain plus the impact of the 
intervention. 

3. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are; a smaller standard deviation on a given measure would indicate that participants 
had more similar outcomes. The standard deviations in the Torgesen et al. (2006) study were the population standard deviations for these standardized outcomes. 

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the control group. 

5. For an explanation of the effect size calculation, see WWC Procedures and Standards Flandbook, Appendix B. 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the control condition. The 

improvement index can take on values between -50 and +50, with positive numbers denoting favorable results for the intervention group. 

8. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple compari- 

sons. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures and Standards Handbook, Appendix C for clustering and WWC Procedures and Standards 
Handbook, Appendix D for multiple comparisons. In the case of Torgesen et al. (2006) and the alphabetics domain, no corrections for clustering were needed because the authors adjusted for 
clustering, and no correction for multiple comparisons was needed because there were no statistically significant findings in this domain. 

9. This row provides the study average, which in this instance is also the domain average. The WWC-computed domain average effect size is a simple average rounded to two decimal places. The 
domain improvement index is calculated from the average effect size. 
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Appendix A3.2 Summary of study findings included in the rating for the reading fluency domain 1 





Authors’ findings from the study 







Mean outcome 2 
(standard deviation) 3 


WWC calculations 


Outcome measure 

Study 

sample 

Sample size 
(school units/ 
students) 

Corrective 

Reading Comparison 

group group 

Mean 

difference 4 
(Corrective 
Reading 
- comparison) 

Statistical 

Effect significance 6 

size 5 (at a = 0.05) 

Improvement 

index 7 



Torgesen et al., 2006 8 





Oral Reading Fluency test Grade 5 7/86 

96.80 91.90 

(47.00) (47.00) 

4.90 

0.10 

ns 

+4 

Domain average for reading fluency (Torgesen et al., 2006) 9 



0.10 

ns 

+4 


ns = not statistically significant 


1. This appendix reports findings considered for the effectiveness rating and the average improvement indices for the reading fluency domain. Follow-up findings from the same studies are not 
included in these ratings but are reported in Appendix A4.2. The study also included subgroup analyses by initial skill level (WRMT-R Word Attack subtest and Peabody Picture Vocabulary Test 
[PPVT]) and socioeconomic status. No differences were found between subgroups of students for the reading fluency outcome. 

2. For Torgesen et al. (2006), the mean outcomes were computed using information reported in the paper. For the control group, the mean outcome is the control group baseline mean standard score 
plus the control group gain. For the intervention group, the mean outcome is the control group baseline mean standard score plus the control group gain plus the impact of the intervention. 

3. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are; a smaller standard deviation on a given measure would indicate that participants 
had more similar outcomes. The standard deviations in the Torgesen et al. (2006) study were the population standard deviations for these standardized outcomes. 

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the control group. 

5. For an explanation of the effect size calculation, see WWC Procedures and Standards Handbook, Appendix B. 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the control condition. The 

improvement index can take on values between -50 and +50, with positive numbers denoting favorable results for the intervention group. 

8. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple compari- 

sons. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures and Standards Handbook, Appendix C for clustering and WWC Procedures and Standards 
Handbook, Appendix D for multiple comparisons. In the case of Torgesen et al. (2006) and the reading fluency domain, no corrections for clustering were needed because the authors adjusted 
for clustering, and no correction for multiple comparisons was needed because there is only one outcome in this domain. 

9. This row provides the study average, which in this instance is also the domain average. The WWC-computed domain average effect size is a simple average rounded to two decimal places. The 
domain improvement index is calculated from the average effect size. 
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Appendix A3.3 Summary of study findings included in the rating for the comprehension domain * 1 





Authors’ findings from the study 







Mean outcome 2 
(standard deviation) 3 


WWC calculations 


Outcome measure 

Study 

sample 

Sample size 
(school units/ 
students) 

Corrective 

Reading Comparison 

group group 

Mean 

difference 4 
(Corrective 
Reading 
- comparison) 

Statistical 

Effect significance 6 

size 5 (at a = 0.05) 

Improvement 

index 7 


Torgesen et al., 2006 8 


WRMT-R: Passage 
Comprehension subtest 

Grade 5 

7/86 

93.80 

(15.00) 

92.00 

(15.00) 

1.80 

0.12 

ns 

+5 

GRADE: Passage 

Grade 5 

7/86 

96.30 

96.00 

0.30 

0.02 

ns 

+1 


Comprehension subtest (15.00) (15.00) 

Domain average for comprehension (Torgesen et al., 2006) 9 0.07 ns +3 

ns = not statistically significant 

WRMT-R = Woodcock Reading Mastery Test-Revised 

GRADE = Group Reading Assessment and Diagnostic Evaluation 

1. This appendix reports findings considered for the effectiveness rating and the average improvement indices for the comprehension domain. Follow-up findings from the same studies are not 
included in these ratings but are reported in Appendix A4.3. The study also included subgroup analyses by initial skill level (WRMT-R Word Attack subtest and Peabody Picture Vocabulary Test 
[PPVT]) and socioeconomic status. No differences were found between subgroups of students for outcomes in the comprehension domain. 

2. For Torgesen et al. (2006), the mean outcomes were computed using information reported in the paper. For the control group, the mean outcome is the control group baseline mean standard score 
plus the control group gain. For the intervention group, the mean outcome is the control group baseline mean standard score plus the control group gain plus the impact of the intervention. 

3. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are; a smaller standard deviation on a given measure would indicate that participants 
had more similar outcomes. The standard deviations in the Torgesen et al. (2006) study were the population standard deviations for these standardized outcomes. 

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the control group. 

5. For an explanation of the effect size calculation, see WWC Procedures and Standards Flandbook, Appendix B. 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the control condition. The 

improvement index can take on values between -50 and +50, with positive numbers denoting favorable results for the intervention group. 

8. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple compari- 

sons. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures and Standards Handbook, Appendix C for clustering and WWC Procedures and Standards 
Handbook, Appendix D for multiple comparisons. In the case of Torgesen et al. (2006) and the comprehension domain, no corrections for clustering were needed because the authors adjusted 
for clustering. No correction for multiple comparisons was needed because there were no statistically significant findings in this domain. 

9. This row provides the study average, which in this instance is also the domain average. The WWC-computed domain average effect size is a simple average rounded to two decimal places. The 
domain improvement index is calculated from the average effect size. 
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Appendix A4.1 


Summary of follow-up findings for the alphabetics domain 1 





Authors’ findings from the study 







Mean outcome 2 
(standard deviation) 3 


WWC calculations 


Outcome measure 

Study 

sample 

Sample size 
(school units/ 
students) 

Corrective 

Reading Comparison 

group group 

Mean 

difference 4 
(Corrective 
Reading 
- comparison) 

Statistical 

Effect significance 6 

size 5 (at a = 0.05) 

Improvement 

index 7 


Torgensen, et al., 2007 8 


WRMT-R: Word 
Attack subtest 

Grade 5 

7/84 

98.60 

(15.00) 

98.40 

(15.00) 

0.20 

0.01 

ns 

+1 

WRMT-R: Word 
Identification subtest 

Grade 5 

7/84 

92.10 

(15.00) 

94.00 

(15.00) 

-1.90 

-0.13 

ns 

-5 

TOWRE: Phonemic Decoding 
Efficiency (PDE) subtest 

Grade 5 

7/84 

90.00 

(15.00) 

88.60 

(15.00) 

1.40 

0.09 

ns 

+4 

TOWRE: Sight Word 
Efficiency (SWE) subtest 

Grade 5 

7/84 

8710 

(15.00) 

87.50 

(15.00) 

-0.40 

-0.03 

ns 

-1 


ns = not statistically significant 

WRMT-R = Woodcock Reading Mastery Test-Revised 

TOWRE = Test of Word Reading Efficiency 


1. This appendix presents findings from data collected one year after the end of the implementation of the intervention for measures that fall in the alphabetics domain. Data that reflected students’ 
exposure to six months of the intervention were used for rating purposes and are presented in Appendix A3.1. Torgesen et al. (2007) also included subgroup analyses by initial skill level (WRMT-R 
Word Attack subtest and Peabody Picture Vocabulary Test [PPVT]) and socioeconomic status. The study reported that Corrective Reading had a statistically significant positive effect on the 
TOWRE PDE and TOWRE SWE for students eligible for free or reduced-price lunch programs. The study also reported that Corrective Reading had a statistically significant positive effect on the 
TOWRE SWE for students with low initial PPVT scores and on the TOWRE PDE for students with low initial Word Attack scores but high initial PPVT scores. Additional subgroup analyses found 
statistically significant positive effects on Word Attack for students with low initial Word Attack scores and for students with low scores on both baseline assessments (Word Attack and PPVT). 
No other differences were reported between subgroups of students for outcomes in the alphabetics domain. 

2. For Torgesen et al. (2007), the mean outcomes were computed using information reported in the paper. For the control group, the mean outcome is the control group baseline mean standard score 
plus the control group gain. For the intervention group, the mean outcome is the control group baseline mean standard score plus the control group gain plus the impact of the intervention. 

3. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are; a smaller standard deviation on a given measure would indicate that participants 
had more similar outcomes. The standard deviations in the Torgesen et al. (2007) study were the population standard deviations for these standardized outcomes. 

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the control group. 

5. For an explanation of the effect size calculation, see WWC Procedures and Standards Flandbook, Appendix B. 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the control condition. The 

improvement index can take on values between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

8. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple compari- 

sons. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures and Standards Handbook, Appendix C for clustering and WWC Procedures and Standards 
Handbook, Appendix D for multiple comparisons. In the case of Torgesen et al. (2007) and the alphabetics domain, no corrections for clustering were needed because the authors adjusted for 
clustering, and no correction for multiple comparisons was needed because there were no statistically significant findings in this domain. 
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Appendix A4.2 Summary of follow-up findings for the reading fluency domain 1 





Authors’ findings from the study 







Mean outcome 2 
(standard deviation) 3 


WWC calculations 


Outcome measure 

Study 

sample 

Sample size 
(school units/ 
students) 

Corrective 

Reading Comparison 

group group 

Mean 

difference 4 
(Corrective 
Reading 
- comparison) 

Statistical 

Effect significance 6 

size 5 (at a = 0.05) 

Improvement 

index 7 





Torgesen et al., 2007 8 





Oral Reading Fluency test 

Grade 5 

7/84 

102.10 107.40 

(47.00) (47.00) 

-5.30 

-0.11 

ns 

-4 


ns = not statistically significant 


1. This appendix presents findings from data collected one year after the end of the implementation of the intervention for measures that fall in the reading fluency domain. Data that reflected stu- 
dents’ exposure to six months of the intervention were used for rating purposes and are presented in Appendix A3. 2. Torgesen et al. (2007) also included subgroup analyses by initial skill level 
(WRMT-R Word Attack subtest and Peabody Picture Vocabulary Test [PPVT]) and socioeconomic status. The study reported that Corrective Reading had a statistically significant negative effect 
on the Oral Reading Fluency test for students with high initial PPVT scores, for students who had both high initial word attack scores and high initial PPVT scores, and for students who were not 
eligible for free or reduced-price lunch programs. No other differences were found between subgroups of students for the reading fluency outcome. 

2. For Torgesen et al. (2007), the mean outcomes were computed using information reported in the paper. For the control group, the mean outcome is the control group baseline mean standard score 
plus the control group gain. For the intervention group, the mean outcome is the control group baseline mean standard score plus the control group gain plus the impact of the intervention. 

3. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are; a smaller standard deviation on a given measure would indicate that participants 
had more similar outcomes. The standard deviations in the Torgesen et al. (2007) study were the population standard deviations for these standardized outcomes. 

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the control group. 

5. For an explanation of the effect size calculation, see WWC Procedures and Standards Handbook, Appendix B. 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the control condition. The 
improvement index can take on values between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

8. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple compari- 
sons. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures and Standards Handbook, Appendix C for clustering and WWC Procedures and Standards 
Handbook, Appendix D for multiple comparisons. In the case of Torgesen et al. (2007) and the reading fluency domain, no corrections for clustering were needed because the authors adjusted 
for clustering, and no correction for multiple comparisons was needed because there is only one outcome in this domain. 
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Appendix A4.3 Summary of follow-up findings for the comprehension domain 1 





Authors’ findings from the study 







Mean outcome 2 
(standard deviation) 3 


WWC calculations 


Outcome measure 

Study 

sample 

Sample size 
(school units/ 
students) 

Corrective 

Reading Comparison 

group group 

Mean 

difference 4 
(Corrective 
Reading 
- comparison) 

Statistical 

Effect significance 6 

size 5 (at a = 0.05) 

Improvement 

index 7 





Torgesen et al., 2007 8 





WRMT-R: Passage 

Grade 5 

7/84 

93.50 

95.60 

-2.10 

-0.14 

ns 

-6 

Comprehension 



(15.00) 

(15.00) 





GRADE: Passage 
Comprehension 

Grade 5 

7/84 

91.60 

(15.00) 

91.60 

(15.00) 

0 

0 

ns 

0 


ns = not statistically significant 

WRMT-R = Woodcock Reading Mastery Test-Revised 

GRADE = Group Reading Assessment and Diagnostic Evaluation 


1. This appendix presents findings from data collected one year after the end of the implementation of the intervention for measures that fall in the comprehension domain. Data that reflected stu- 
dents’ exposure to six months of the intervention were used for rating purposes and are presented in Appendix A3. 3. Torgesen et al. (2007) also included subgroup analyses by initial skill level 
(WRMT-R Word Attack subtest and Peabody Picture Vocabulary Test [PPVT]) and socioeconomic status. The study reported that Corrective Reading had a statistically significant negative effect 
on both comprehension outcomes for students with high initial level of Word Attack scores. The difference was also negative and statistically significant on the GRADE outcome for students 
with high initial level of Word Attack and PPVT scores, but positive and statistically significant for students with low initial PPVT scores. The study also found a statistically significant positive 
effect on the WRMT Passage Comprehension for students with low skill level on both baseline assessments: Word Attack and PPVT. No other differences were reported between subgroups of 
students for outcomes in the comprehension domain. 

2. For Torgesen et al. (2007), the mean outcomes were computed using information reported in the paper. For the control group, the mean outcome is the control group baseline mean standard score 
plus the control group gain. For the intervention group, the mean outcome is the control group baseline mean standard score plus the control group gain plus the impact of the intervention. 

3. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are; a smaller standard deviation on a given measure would indicate that participants 
had more similar outcomes. The standard deviations in the Torgesen et al. (2007) study were the population standard deviations for these standardized outcomes. 

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the control group. 

5. For an explanation of the effect size calculation, see WWC Procedures and Standards Flandbook, Appendix B. 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the control condition. The 

improvement index can take on values between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

8. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple compari- 

sons. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures and Standards Flandbook, Appendix C for clustering and WWC Procedures and Standards 
Flandbook, Appendix D for multiple comparisons. In the case of Torgesen et al. (2007) and the comprehension domain, no corrections for clustering were needed because the authors adjusted 
for clustering. No correction for multiple comparisons was needed because there were no statistically significant findings in this domain. 
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Appendix A5.1 Corrective Reading rating for the alphabetics domain 

The WWC rates an intervention’s effects for a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. 1 
For the outcome domain of alphabetics, the WWC rated Corrective Reading as having no discernible effects for adolescent learners. 

Rating received 

No discernible effects: No affirmative evidence of effects. 

• Criterion 1: No studies showing a statistically significant or substantively important effect, either positive or negative. 

Met. No studies showed a statistically significant or substantively important effect, either positive or negative. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No studies showed a statistically significant positive effect. 

AND 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. No studies showed a statistically significant or substantively important negative effect. 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect. 

Not met. No studies showed a statistically significant or substantively important positive effect. 

AND 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect and fewer or the same number of studies showing 
indeterminate effects than showing statistically significant or substantively important positive effects. 

Not met. No studies showed a statistically significant or substantively important negative effect. One study showed indeterminate effects. 

Mixed effects: Evidence of inconsistent effects as demonstrated through either of the following criteria. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect, and at least one study showing a statistically significant 
or substantively important negative effect, but no more such studies than the number showing a statistically significant or substantively important positive effect. 

Not met. No studies showed a statistically significant or substantively important positive effect, and no studies showed a statistically significant or 
substantively important negative effect. 

OR 

• Criterion 2: At least one study showing a statistically significant or substantively important effect, and more studies showing an indeterminate effect than showing 
a statistically significant or substantively important effect. 

Not met. No studies showed a statistically significant or substantively important effect. One study showed indeterminate effects. 


(continued) 
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Appendix A5.1 Corrective Reading rating for the alphabetics domain (continued) 


Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: One study showing a statistically significant or substantively important negative effect and no studies showing a statistically significant or substantively 
important positive effect. 

Not met. No studies showed a statistically significant or substantively important effect, either positive or negative. 

OR 

• Criterion 2: Two or more studies showing statistically significant or substantively important negative effects, at least one study showing a statistically significant 
or substantively important positive effect, and more studies showing statistically significant or substantively important negative effects than showing statistically 
significant or substantively important positive effects. 

Not met. No studies showed a statistically significant or substantively important effect, either positive or negative. 

Negative effects: Strong evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant negative effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No studies showed a statistically significant negative effect. 

AND 

• Criterion 2: No studies showing statistically significant or substantively important positive effects. 

Met. No studies showed a statistically significant or substantively important positive effect. 

1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings of 
potentially positive or potentially negative effects. For a complete description, see the WWC Procedures and Standards Handbook, Appendix E. 
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Appendix A5.2 Corrective Reading rating for the reading fluency domain 

The WWC rates an intervention’s effects for a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. 1 
For the outcome domain of reading fluency, the WWC rated Corrective Reading as having no discernible effects for adolescent learners. 

Rating received 

No discernible effects: No affirmative evidence of effects. 

• Criterion 1: No studies showing a statistically significant or substantively important effect, either positive or negative. 

Met. No studies showed a statistically significant or substantively important effect, either positive or negative. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No studies showed a statistically significant positive effect. 

AND 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. No studies showed a statistically significant or substantively important negative effect. 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect. 

Not met. No studies showed a statistically significant or substantively important positive effect. 

AND 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect and fewer or the same number of studies showing 
indeterminate effects than showing statistically significant or substantively important positive effects. 

Not met. No studies showed a statistically significant or substantively important negative effect. One study showed indeterminate effects. 

Mixed effects: Evidence of inconsistent effects as demonstrated through either of the following criteria. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect, and at least one study showing a statistically significant 
or substantively important negative effect, but no more such studies than the number showing a statistically significant or substantively important positive effect. 

Not met. No studies showed a statistically significant or substantively important positive effect, and no studies showed a statistically significant or 
substantively important negative effect. 

OR 

• Criterion 2: At least one study showing a statistically significant or substantively important effect, and more studies showing an indeterminate effect than showing 
a statistically significant or substantively important effect. 

Not met. No studies showed a statistically significant or substantively important effect. One study showed indeterminate effects. 


(continued) 
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Appendix A5.2 Corrective Reading rating for the reading fluency domain (continued) 


Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: One study showing a statistically significant or substantively important negative effect and no studies showing a statistically significant or substantively 
important positive effect. 

Not met. No studies showed a statistically significant or substantively important effect, either positive or negative. 

OR 

• Criterion 2: Two or more studies showing statistically significant or substantively important negative effects, at least one study showing a statistically significant 
or substantively important positive effect, and more studies showing statistically significant or substantively important negative effects than showing statistically 
significant or substantively important positive effects. 

Not met. No studies showed a statistically significant or substantively important effect, either positive or negative. 

Negative effects: Strong evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant negative effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No studies showed a statistically significant negative effect. 

AND 

• Criterion 2: No studies showing statistically significant or substantively important positive effects. 

Met. No studies showed a statistically significant or substantively important positive effect. 

1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings of 
potentially positive or potentially negative effects. For a complete description, see the WWC Procedures and Standards Handbook, Appendix E. 
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Appendix A5.3 Corrective Reading rating for the comprehension domain 

The WWC rates an intervention’s effects for a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. 1 
For the outcome domain of comprehension, the WWC rated Corrective Reading as having no discernible effects for adolescent learners. 

Rating received 

No discernible effects: No affirmative evidence of effects. 

• Criterion 1: No studies showing a statistically significant or substantively important effect, either positive or negative. 

Met. No studies showed a statistically significant or substantively important effect, either positive or negative. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No studies showed a statistically significant positive effect. 

AND 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. No studies showed a statistically significant or substantively important negative effect. 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect. 

Not met. No studies showed a statistically significant or substantively important positive effect. 

AND 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect and fewer or the same number of studies showing 
indeterminate effects than showing statistically significant or substantively important positive effects. 

Not met. No studies showed a statistically significant or substantively important negative effect. One study showed indeterminate effects. 

Mixed effects: Evidence of inconsistent effects as demonstrated through either of the following criteria. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect, and at least one study showing a statistically significant 
or substantively important negative effect, but no more such studies than the number showing a statistically significant or substantively important positive effect. 

Not met. No studies showed a statistically significant or substantively important positive effect, and no studies showed a statistically significant or 
substantively important negative effect. 

OR 

• Criterion 2: At least one study showing a statistically significant or substantively important effect, and more studies showing an indeterminate effect than showing 
a statistically significant or substantively important effect. 

Not met. No studies showed a statistically significant or substantively important effect. One study showed indeterminate effects. 


(continued) 
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Appendix A5.3 Corrective Reading rating for the comprehension domain (continued) 


Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: One study showing a statistically significant or substantively important negative effect and no studies showing a statistically significant or substantively 
important positive effect. 

Not met. No studies showed a statistically significant or substantively important effect, either positive or negative. 

OR 

• Criterion 2: Two or more studies showing statistically significant or substantively important negative effects, at least one study showing a statistically significant 
or substantively important positive effect, and more studies showing statistically significant or substantively important negative effects than showing statistically 
significant or substantively important positive effects. 

Not met. No studies showed a statistically significant or substantively important effect, either positive or negative. 

Negative effects: Strong evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant negative effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No studies showed a statistically significant negative effect. 

AND 

• Criterion 2: No studies showing statistically significant or substantively important positive effects. 

Met. No studies showed a statistically significant or substantively important positive effect. 

1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings of 
potentially positive or potentially negative effects. For a complete description, see the WWC Procedures and Standards Handbook, Appendix E. 
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Appendix A6 Extent of evidence by domain 


Outcome domain 

Number of studies 

Sample size 

Schools 

Students 

Extent of evidence 1 

Alphabetics 

1 

7 school units 2 

86 

Small 

Reading fluency 

1 

7 school units 2 

86 

Small 

Comprehension 

1 

7 school units 2 

86 

Small 

General literacy achievement 

na 

na 

na 

na 


na = not applicable/not studied 

1. A rating of “medium to large” requires at least two studies and two schools across studies in one domain and a total sample size across studies of at least 350 students or 14 classrooms. Other- 
wise, the rating is “small.” For more details on the extent of evidence categorization, see the WWC Procedures and Standards Handbook, Appendix G. 

2. A school unit consists of several schools and includes two 3rd-grade and two 5th-grade instructional groups. The exact number of schools participating in Corrective Reading is unknown. 
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